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Abstract. It is well-known that constructing models of higher-order 
probabilistic programming languages is challenging. We show how to 
construct step-indexed logical relations for a probabilistic extension of 
a higher-order programming language with impredicative polymorphism 
and recursive types. We show that the resulting logical relation is sound 
and complete with respect to the contextual preorder and, moreover, 
that it is convenient for reasoning about concrete program equivalences. 
Finally, we extend the language with dynamically allocated first-order 
references and show how to extend the logical relation to this language. 
We show that the resulting relation remains useful for reasoning about 
examples involving both state and probabilistic choice. 


1 Introduction 

It is well known that it is challenging to develop techniques for reasoning about 
programs written in probabilistic higher-order programming languages. A prob¬ 
abilistic program evaluates to a distribution of values, as opposed to a set of 
values in the case of nondeterminism or a single value in the case of determinis¬ 
tic computation. Probability distributions form a monad. This observation has 
been used as a basis for several denotational domain-theoretic models of proba¬ 
bilistic languages and also as a guide for designing probabilistic languages with 
monadic types |l5l2ll2()| . Game semantics has also been used to give models 
of probabilistic programming languages [SHU and a fully abstract model using 
coherence spaces for PCF with probabilistic choice was recently presented [T^ . 

The majority of models of probabilistic programming languages have been 
developed using denotational semantics. However, Johann et.al. US] developed 
operationally-based logical relations for a polymorphic programming language 
with effects. Two of the effects they considered were probabilistic choice and 
global ground store. However, as pointed out by the authors US], extending their 
construction to local store and, in particular, higher-order local store, is likely to 
be problematic. Recently, operationally-based bisimulation techniques have been 
extended to probabilistic extensions of PCF m- The operational semantics of 
probabilistic higher-order programming languages has been investigated in m- 

Step-indexed logical relations m have proved to be a successful method for 
proving contextual approximation and equivalence for programming languages 
with a wide range of features, including computational effects. 






2 


In this paper we show how to extend the method of step-indexed logical rela¬ 
tions to reason about contextual approximation and equivalence of probabilistic 
higher-order programs. To define the logical relation we employ biorthogonal¬ 
ity mm and step-indexing. Biorthogonality is used to ensure completeness of 
the logical relation with respect to contextual equivalence, but it also makes it 
possible to keep the value relations simple, see Fig. [T] Moreover, the definition 
using biorthogonality makes it possible to “externalize” the reasoning in many 
cases when proving example equivalences. By this we mean that the reasoning 
reduces to algebraic manipulations of probabilities. This way, the quantitative 
aspects do not complicate the reasoning much, compared to the usual reason¬ 
ing with step-indexed logical relations. To define the biorthogonal lifting we use 
two notions of observation; the termination probability and its stratified version 
approximating it. We define these and prove the required properties in Section|3l 

We develop our step-indexed logical relations for the call-by-value language 
This is system F with recursive types, extended with a single probabilistic 
choice primitive rand. The primitive rand takes a natural number n and reduces 
with uniform probability to one of 1, 2,..., n. Thus randn represents the uniform 
probability distribution on the set {1, 2,..., n}. We choose to add rand instead 
of just a single coin flip primitive to make the examples easier to write. 

To show that the model is useful we use it to prove some example equiva¬ 
lences in Section [5] We show two examples based on parametricity. In the first 
example, we characterize elements of the universal type Va.a —>■ a. In a de¬ 
terministic language, and even in a language with nondeterministic choice, the 
only interesting element of this type is the identity function. However, since in a 
probabilistic language we not only observe the end result, but also the likelihood 
with which it is returned, it turns out that there are many more elements. Con¬ 
cretely, we show that the elements of the type Va.a a that are of the form 
Aa.Xx.e, correspond precisely to left-computable real numbers in the interval 
[0,1]. In the second example we show a free theorem involving functions on lists. 
We show additional equivalences in the Appendix, including the correctness of 
von Neumann’s procedure for generating a fair sequence of coin tosses from an 
unfair coin, and equivalences from the recent papers using bisimulations m- 

We add dynamically allocated references to the language and extend the 
logical relation to the new language in Section |6l For simplicity we only sketch 
how to extend the construction with first-order state. This already suggests that 
an extension with general references can be done in the usual way for step- 
indexed logical relations. We conclude the section by proving a representation 
independence result involving both state and probabilistic choice. 

All the references to the Appendix in this paper refer to appendix in the 
online version [B]. 

2 The language F'^’® 

The language is a standard pure functional language with recursive, universal 
and existential types with an additional choice primitive rand. The base types 
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include the type of natural numbers nat with some primitive operations. The 
grammar of terms e is 

e ::= cc | () | rande | n \ if i e then ei else 62 | P e | S e | (ei, 62 ) | proj^ e 
I Xx.e I ei 62 I ini e | inr e | match (e, xi.ei, 0 : 2 . 62 ) | A.e \ e[] 

I packe I unpack Ci as a; in 62 | folde | unfolde 

We write n for the numeral representing the natural number n and S and P are 
the successor and predecessor functions, respectively. For convenience, numerals 
start at 1. Given a numeral n, the term randn evaluates to one of the numerals 
n with uniform probability. There are no types in the syntax of terms, 
e.g., instead of Aa.e and er we have A.e and e[]. This is for convenience only. 

We write a, P,... for type variables and x,y,... for term variables. The no¬ 
tation t[t / d] denotes the simultaneous capture-avoiding substitution of types t 
for the free type variables a in the type r; e[v/x\ denotes simultaneous capture¬ 
avoiding substitution of values v for the free term variables x in the term e. 

We write Stk for the set of evaluation contexts given by the call-by-value 
reduction strategy. Given two evaluation contexts E, E' we dehne their composi¬ 
tion EoE' by induction on E in the natural way. Given an evaluation context E 
and expression e we write E[e] for the term obtained by plugging e into E. For 
any two evaluation contexts E and E' and a term e we have E[E' [e]] = {EoE')[e]. 

For a type variable context A, the judgment A\- t expresses that the free 
type variables in r are included in A. The typing judgments are entirely standard 
with the addition of the typing of rand which is given by the rule 

A \ r \- e \ nat 
A \ r \- rand e : nat 

The complete set of typing rules are in the Appendix. We write d{A) for the 
set of types well-formed in context Z\, and T for the set of closed types r. We 
write Val (r) and Tm (r) for the sets of closed values and terms of type r, 
respectively. We write Val and Tm for the set of closed values and closed 
terms, respectively. Stk (r) denotes the set of r-accepting evaluation contexts, 
i.e., evaluation contexts E, such that given any closed term e of type r, E[e] is 
a typeable term. Stk denotes the set of all evaluation contexts. 

For a typing context E = xi'.ti, ..., Xn'-Tn with ti, ..., t„ G T, let Subst(T) 
denote the set of type-respecting value substitutions, i.e. for all i, ^(xi) G 
Val (Ti). In particular, if Z\ | T h e : r then 0 | 0 h ey : rJ for any 5 G and 
7 G Subst(T^), and the type system satisfies standard properties of progress 
and preservation and a canonical forms lemma. 

The operational semantics of the language is a standard call-by-value seman¬ 
tics but weighted with p G [0,1] which denotes the likelihood of that reduction. 
We write -S? for the one-step reduction relation. All the usual /3 reductions have 
weight equal to 1 and the reduction from randn is 

^ In particular, we do not require them to be typeable. 
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randn A k for k G {1, 2 ,..., n}. 

The rest of the rules are given in Fig.[5]in the Appendix. The operational seman¬ 
tics thus gives rise to a Markov chain with closed terms as states. In particular 
for each term e we have ,, p , p < 1. 

^-^e' \ e~-+e' ^ — 

3 Observations and biorthogonality 

We will use biorthogonality to define the logical relation. This section provides 
the necessary observation predicates used in the definition of the biorthogonal 
lifting of value relations to expression relations. Because of the use of biorthogo¬ 
nality the value relations (see Fig. [1]) remain as simple as for a language without 
probabilistic choice. The new quantitative aspects only appear in the definition 
of the biorthogonal lifting (TT-closure) defined in Section 2] Two kinds of ob¬ 
servations are used. The probability of termination, (e), which is the actual 
probability that e terminates, and its approximation, the stratified termination 
probability (e), where fc S N denotes, intuitively, the number of computation 
steps. The stratified termination probability provides the link between steps in 
the operational semantics and the indexing in the definition of the interpretation 
of types. 

The probability of termination, tpl*' (•), is a function of type Tm —> I where 
X is the unit interval [0,1]. Since X is a pointed w-cpo for the usual order, so is 
the space of all functions Tm —>■ X with pointwise ordering. We define (•) as 
a fixed point of the continuous function (p on this w-cpo: Let J- = Tm X and 
define : X” —?► X" as 

{ 1 if e G Val 

p ■ f (e') otherwise 

e-^e' 

Note that if e is stuck then <P{f){e) = 0 since the empty sum is 0. 

The function <P is monotone and preserves suprema of w-chains. The proof is 
straightforward and can be found in the Appendix. Thus <P has a least fixed point 
in X and we denote this fixed point by tpl*' (•), i.e., fP'*^ (e) = sup„g^ <?"(T)(e). 

To define the stratified observations we need the notion of a path. Given 
terms e and e' a path tt from e to e', written tt : e e', is a sequence e ei % 
g 2 £4 • ■ • e'. The weight 211 (tt) of a path tt is the product of the weights of 

reductions in tt. We write 91 for the set of all paths and • for their concatenation 
(when defined). For a non-empty path tt G 91 we write ^ (tt) for its last expression. 
We call reductions of the form unf old(foldu) v unfold-fold reductions 

and reductions of the form randn A k choice reductions. If none of the reduc¬ 
tions in a path tt is a choice reduction we call tt choice-free and similarly if none 
of the reductions in tt is an unfold-fold reductions we call tt unfold-fold free. 

We define the following types of multi-step reductions which we use in the 
definition of the logical relation. 
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e' if there is a choice-free path from e to e' 

e' if there is an unfold-fold free path from e to e'. 

e' if e e' and e e'. 


The following useful lemma states that all but choice reductions preserve 
the probability of termination. As a consequence, we will see that all but choice 
reductions preserve equivalence. 

Lemma 3.1. Let e,e' e Tm and e e!. Then tpl*' (e) = (e'). 

The proof proceeds on the length of the reduction path with the strengthened 
induction hypothesis stating that the probabilities of termination of all elements 
on the path are the same. To define the stratified probability of termination that 
approximates (•) we need an auxiliary notion. 

Definition 3.2. For a closed expression e € Tm we define Red (e) as the 
(unique) set of paths containing exactly one unfold-fold or choice reduction 
and ending with such a reduction. More precisely, we define the function Red : 
Tm —>■ V (91) as the least function satisfying 


Red (e) = < 


{e e'} 

{e E[k] I p 



= {1,2,...,n}} 

TT G Red (e')| 


if e = A[unf old (f oldu)] 
if e = A[rELndn] 
if e ^ e' and e e' 
otherwise 


where we order the power set V (91) by subset inclusion. 

Using Red (•) we define a monotone map F ■. T ^ T that preserves ca-chains. 

f 1 if G Val, e ^ u 

^(/)(e) = < ^ HO (tt) ■ f {£ (n)) otherwise 

7rGR.ed(e) 

and then define fp^ (e) = <f'^(T)(e). The intended meaning of (e) is the 
probability that e terminates within k unfold-fold and choice reductions. Since 
F is monotone we have that fp^ (e) < fP^+i (e) for any k and e. 

The following lemma is the reason for counting only certain reductions, 
cf. [in]. It allows us to stay at the same step-index even when taking steps in 
the operational semantics. As a consequence we will get a more extensional log¬ 
ical relation. The proof is by case analysis and can be found in the Appendix. 


Lemma 3.3. Let e,e' G Tm. If e e' then for all k, tp^ (e) = fp^ (e'). 


The following is immediate from the definition of the chain 
the fact that randn reduces with uniform probability. 



and 
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Lemma 3.4. Let e be a closed term. Ife-^e' and the reduction is an unfold-fold 
reduction then (e) = (e'). If the reduction from e is a choice reduction, 

then (e) = S 7 i-GRed(e) (’’’))■ 

The following proposition is needed to prove adequacy of the logical relation 
with respect to contextual equivalence. It is analogous to the property used to 
prove adequacy of step-indexed logical relations for deterministic and nondeter- 
ministic languages. Consider the case of may-equivalence. To prove adequacy in 
this case (cf. [U Theorem 4.8]) we use the fact that if e may-terminates, then 
there is a natural number n such that e terminates in n steps. This property 
does not hold in the probabilistic case, but the property analogous to it that is 
sufficient to prove adequacy still holds. 

Proposition 3.5. For eaeh e S Tm we have (e) < sup;,g^ (e)^. 

Proof. We only give a sketch; the full proof can be found in the Appendix. We 
use Scott induction on the set 5 = |/ G J" | Ve, /(e) < sup^g,^ ^‘P^ (e)^ |. It is 
easy to see that S is closed under limits of w-chains and that T G 5 so we only 
need to show that S is closed under We can do this by considering the kinds 
of reductions from e when considering <?(/)(e) for / G 5. 

4 Logical, CIU and contextual approximation relations 

The contextual and CIU (closed instantiations of uses [18]) approximations are 
defined in a way analogous to the one for deterministic programming languages. 
We require some auxiliary notions. A type-indexed relation 7?. is a set of tuples 
(Z\, r, e, e', t) such that ALT and Z\ h r and A \ F \- e : t and A \ F \- e' : t. 
We write A \ F \- e TZ e' : t for {A, F, e, e', t) G TZ. 

Definition 4.1 (Precongruence). A type-indexed relation TZ is reflexive if 
A \ F \- e '. T implies A\F\-eTZe:T. It is transitive if A \ F \- e IZ e' : t and 
A \ F \- e' TZ e" : T implies A \ F \- e TZ e" : t. It is compatible if it is closed 
under the term forming rules, e.^.J^ 

A I F, x:ti \- eTZ e' : T 2 A \ F \- eTZ e' : nat 

A \ F \- Xx.e TZ Xx.e' : ti T 2 A \ F \- rande TZ rande^ : nat 

A precongruence is a reflexive, transitive and compatible type-indexed relation. 

The compatibility rules guarantee that a compatible relation is sufficiently 
big, i.e., at least reflexive. In contrast, the notion of adequacy, which relates the 
operational semantics with the relation, guarantees that it is not too big. In the 
deterministic case, a relation TZ is adequate if when eTZ e' are two related closed 
terms, then if e terminates so does e'. Here we need to compare probabilities of 
termination instead, since these are our observations. 

We only show a few rules, the rest are analogous and can be found in the Appendix. 
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Definition 4.2. A type-indexed relation TZ is adequate if for all e,e' such that 
0 \ 0 \- e TZ e' : T we have (e) < (e'). 

The contextual approximation relation, written Z\ | T h e e' : r, is defined 
as the largest adequate precongruence and the CIU approximation relation, writ¬ 
ten Z\ I T h e e' : r, is defined using evaluation contexts in the usual 

way, e.g. [18], using ipl*' (•) for observations. The fact that the largest adequate 
precongruence exists is proved as in [T8] . 

Logical relation We now define the step-indexed logical relation. We present 
the construction in the elementary way with explicit indexing instead of using a 
logic with guarded recursion as in [10] to remain self-contained. 

Interpretations of types will be defined as decreasing sequences of relations 
on typeable values. For closed types r and a we define the sets VRel (r,tT), 
SRel (r, a) and TRel (r, a) to be the sets of decreasing sequences of relations 
on typeable values, evaluation contexts and expressions respectively. The types r 
and tr denote the types of the left-hand side and the right-hand side respectively, 
i.e. if (u, u) € ipin) for g) € VRel (r, tr) then v has type r and u has type tr. The 
order relation < on these sets is defined pointwise, e.g. for ip,ip G VRel (r, a) 
we write </? < "0 if Vn G N,ip(n) C '0(n). We implicitly use the inclusion from 
VRel (r, cr) to TRel (t, cr). The reason for having relations on values and terms 
of different types on the left and right-hand sides is so we are able to prove 
parametricity properties in Section [S] 

We define maps -J^ : VRel (t, cr) —> SRel (r, cr) and : SRel (r, cr) ^ 
TRel (r, cr). We usually omit the type indices when they can be inferred from 
the context. The maps are defined as follows 

rJjn) = {(E,E') \ Vk < n,\fiv,v') G r{k),^iUE[v]) < {E'[v'])} 

and r^^(n) = {(e,e') | Vfc < n,\/{E,E') G r{k),^l {E[e]) <^^{E'[e'])^ . Note 
that we only count steps evaluating the left term in defining r^ and r-^. We write 
J.TT _ j,T Jqj. their composition from VRel (r, cr) to TRel (r, a). The function 
is order-reversing and is order-preserving and inflationary. 

Lemma 4.3. Let t, a be closed types and r,s G VRel (r, cr). Then r < r^ and 
if r < s then and r^ < . 

For a type-variable context A we define VRel (A) using VRel (•, •) as 

VRel(Z\) = {{ipi,(f2,g3r) \ e T^,Va G A,ipria) G VRel( 1 ^ 1 ( 0 ;),(/ 92 (a))} 

where the first two components give syntactic types for the left and right hand 
sides of the relation and the third component is a relation between those types. 

The interpretation of types, |- h -jl is by induction on the judgement Z\ h r. 

For a judgment Z\ h r and (p G VRel (Z\) we have |Z\ h r] ((/?) G VRel [pi (r), 'P 2 {j)) 
where the ipi and ip 2 are the first two components of ip and Pi{t) denotes sub¬ 
stitution. Moreover !•] is non-expansive in the sense that |Z\ h t] (p)(n) can 


depend only on the values of Lpr{oi){k) for k < n, see [5] for this metric view of 
step-indexing. The interpretation of types is defined in Fig. [TJ Observe that the 
value relations are as simple as for a language without probabilistic choice. The 
crucial difference is hidden in the TT-closure of value relations. 

\A h nat] {ip){n) = {{k,k) | fe € N, fe > 0} 

|[Z\ h r o-] {ip){n) = {(Xx.e,Xy.e') | Vj < n,\/{v,v') G |Z\ h r] 

{{Xx.e)v,{Xy.e')v') G [zi h a] (¥>)^(i)} 
fA h Vq.t] {<p){n) = {(T.e, A.e') \ Vcr, o' £ £ VRel (cr, o'), 

(e, e') G |IZ\, a h r| ((/? [a i->- r])^(n)} 

|Zl h 3a.r]| ((/p)(n) = {(pack a, pack | Ba, cr^ G T, 3r G VRel (cr, cr^), 

{v, v') £lA,a\- t] ((/p [q? !->■ r]) (n)} 

fA h ya.rj (y>)(0) = Val x Val {(p 2 {y-a-T)) 

|Zi h/ro.rl ((/p)(n 3-1) = {(fold a, fold | 

(w, v') £lA,a\- r| (y) [a i-)- |Z\ h ya.rj (y;)]) (n)} 
Fig. 1. Interpretation of types. The cases for sum and product types are in Appendix. 

Context extension lemmas To prove soundness and completeness we need lem¬ 
mas stating how extending evaluation contexts preserves relatedness. We only 
show the case for rand. The rest are similarly simple. 

Lemma 4.4. Let n £N. If {E,E') £ |Z\ h nat] (ip)^{n) are related evaluation 
contexts then (E o (rand []),£!' o (rand[])) G |Z\ b nat] (y))^(n). 

Proof. Let n G N and {v,v') G |Z\ h t] {ip){n). By construction we have v = v' = 
m for some m G N, m > 1. Let fc < n. If fc = 0 the result is immediate, so assume 
k = i + 1. Using Lemma [331 we have (i?[rand27i]) = ^ (-^[1]) and 

using the assumption {E,E') G |Z\ h nat] (y))^(n), the fact that k < n and 
monotonicity in the step-index the latter term is less than 
which by definition of (•) is equal to fpi^ (i3'[randm]). 

We define the logical approximation relation for open terms given the inter¬ 
pretations of types in Fig. [TJ We define Z\ | U h e e' : r to mean 

Vn G N,Vy) G VRel (Z\) ,V( 7 , 7 ') G |Z\ h T] {ip){n), {ej,e'j) G |Z\ h r] {n) 

Here |Zi h U] is the obvious extension of interpretation of types to interpretation 
of contexts which relates substitutions, mapping variables to values. We have 

Proposition 4.5 (Fundamental property). The logical approximation rela¬ 
tion is compatible. In particular it is reflexive. 

Proof. The proof is a simple consequence of the context extension lemmas. We 
show the case for raind. We have to show that A \ P e e' : nat implies 
A \ P \- rand e rand e' : nat. Let n G N, y? G VRel (zl) and ( 7 , 7 ^ £ 
|Z\ h P} {(p){n). Let / = 67 and /' = e' 7 '. Then our assumption gives us (/, /') G 
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|Z\ h nat] ((p)^(n) and we are to show (rand /, rand /') € |Z\ h nat| ((/ 5 )^(n). 
Let j < n and {E,E') G |Z\ h nat| Then from Lemma 14.41 we have 

(E o (rand []), E' o (rand [])) G |Z\ h nat| ((p)^(j) which suffices by the definition 
of the orthogonality relation and the assumption (/, /') G |Z\ h nat| (ip)^(n). 

We now want to relate logical, CIU and contextual approximation relations. 


Corollary 4.6. Logical approximation relation is adequate. 

Proof. Assume 0 | 0 h e e' : r. We are to show that CP'*^ (e) < (e'). 

Straight from the definition we have Vn G N, (e, e') G |0 h T]^(n). The empty 
evaluation context is always related to itself (at any type). This implies Vn G 
N, fp)( (e) < (e') which further implies (since the right-hand side is indepen¬ 

dent of n) that sup„g,^ (*P)( (e)) < ‘P'*^ (e'). Using Proposition 13.51 we thus have 
fpl*' (e) < sup„g,^ (‘P)) (e)) < fP'^ (e') concluding the proof. 

We now have that the logical relation is adequate and compatible. This does 
not immediately imply that it is contained in the contextual approximation 
relation, since we do not know that it is transitive. However we have the following 
lemma where by transitive closure we mean that for each Z\, E and r we take 
the transitive closure of the relation {(e, e') \ A \ E \- e e' : r}. This is 
another type-indexed relation. 

Lemma 4.7. The transitive closure of is compatible and adequate. 

Proof. Transitive closure of an adequate relation is adequate. Similarly the tran¬ 
sitive closure of a compatible and reflexive relation (in the sense of Definition 14. II) 
is again compatible (and reflexive). 

Theorem 4.8 (CIU theorem). The relations coincide. 

Proof. It is standard (e.g. [18]) that is included in We show that the 

logical approximation relation is contained in the CIU approximation relation 
in the standard way for biorthogonal step-indexed logical relations. To see that 
is included in we have by Lemma 14.71 that the transitive closure of 
is an adequate precongruence, thus included in And is included in the 
transitive closure of Corollary lA. 131 in the appendix completes the cycle of 

inclusions. 

Using the logical relation and Theorem 14.81 we can prove some extensionality 
properties. The proofs are standard and can be found in the Appendix. 

Lemma 4.9 (Functional extensionality for values). Suppose T,a G ‘^{A) 
and let f and f be two values of type t ^ a in eontext A \ P. If for all 
u G Val (r) we have A \ P \- f u f'u:a then Z\ | U h / f':T^a. 

The extensionality for expressions, as opposed to only values, of function type 
does not hold in general due to the presence of choice reductions. See Remark [5^ 
for an example. We also have extensionality for values of universal types. 
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Lemma 4.10 (Extensionality for the universal type). Let r G T(Z\,a) be 
a type. Let /,/' be two values of type \/a.T in context A \ L. If for all closed 
types a we have A \ L \- fW /'[] : t\<j lo\ then A \ F \- f f : \/a.T. 

5 Examples 

We now use our logical relation to prove some example equivalences. We show 
two examples involving polymorphism. In the Appendix we show additional 
examples. In particular we show the correctness of von Neumann’s procedure for 
generating a fair sequence of coin tosses from an unfair coin. That example in 
particular shows how the use of biorthogonality allows us to “externalize” the 
reasoning to arithmetic manipulations. 

We first define fix : Va, /3.{(q! —t/3) —>-( 0 ! —t/?)) —?> (a— >-/3) be the term 
A.A.Xf.Xz.Sf{foldSf) z where 6f is the term Ay.let y' = unf oldy in / {Xx.y' y x) 
This is a call-by-value fixed-point combinator. We also write ei ©62 for the term 
if 1 rand2 then ei else 62 . Note that the choice is made before evaluating e^’s. 

We characterize inhabitants of a polymorphic type and show a free theorem. 
For the former, we need to know which real numbers can be probabilities of 
termination of programs. Recall that a real number r is left-computable if there 
exists a computable increasing (not necessarily strictly) sequence {ynlnei^ of 
rational numbers such that r = sup„g^ qn- In Appendix iBl we prove 

Proposition 5.1. For any expression e, (e) is a left-computable real number 
and for any left-computable real number r in the interval [0,1] there is a closed 
term er of type 1 —>• 1 such that (cr ()) = r. 


Inhabitants of the type Va.o; —>■ a In this section we use further syntactic 
sugar for sequencing. When e, e' G Tm are closed terms we write e; e' for (A_.e') e, 
i.e. first run e, ignore the result and then run e'. We will need the property that 
for all terms e,e' S Tm, (e) ■ *P''^(e'). The proof is by Scott 

induction and can be found in the Appendix. 

Using Proposition 15.II we have for each left-computable real r in the interval 
[ 0 , 1 ] an inhabitant F of the type Vo.a ^ a given by A.Xx.Cr {)',x. 

We now show that these are the only inhabitants of Vo.a —>■ a of the form 
A.Xx.e. Given such an inhabitant let r = *P''^ (e[{)/a;]). We know from Proposi¬ 
tion o that r is left-computable. 

Given a value v of type r and n S N we define relations R{n) = {(0,?^)} 
and S{n) = {(u, ())}. Note that the relations are independent of n, i.e. R and 
S are constant relations. By reflexivity of the logical relation and the relational 
actions of types we have 

\/n,{e[0/x\,e[v/x]) G li^{n) and \/n,{e[v/x\,e[f)/x]) G {n) (1) 

from which we conclude that fP'^ (e[()/a;]) = fP'^ {e[v/x\). We now show that v 
and e[v/x\ are GlU-equivalent. Let E G Stk (r) be an evaluation context. Let q = 
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Define the evaluation context E' = —;eq{). Then {E,E') G (n) 
for all n which then means, using m and ProDOsition l3.51 that {E[e[v /A]) < 

{E'[e[Q/x]]). We then have 

{E'[e[{)/x]]) = qj* {e[{)/x]) • (e, ()) = r • {E[v]) 

and SO ‘P'*^ {E[e[vlx\\) < r ■ ipll {E[v]). 

Similarly we have {E',E) G (n) for all n which implies *P''^ (£’[e[u/a;]]) > 
‘P'^ {E'[e[{)/x]]). We also have *P''^ {E'[e[{)/x]]) = r ■ iP'^ (-^[u]). 

So we have proved ‘P'^J' {E[e[v/x\\) = r ■ iP"*^ (£^b]) = (-^b])- 

It is easy to show by Scott induction, that *pi*' (i?[tr-0 'f^]) = (ci- 0) (^H)- 

We have thus shown that for any value u, the terms e[u/a;] and iP'*! (t^Q v) are 
ClU-equivalent. Using Theorem 14.81 and Lemmas 14. 101 and 14.91 we conclude that 
the terms Va.Xx.e and U are contextually equivalent. 


Remark 5.2. Unfortunately we cannot so easily characterize general values of 
the type Va.a —>■ a, that is, those not of the form A.v for a value v. Consider 
the term A.ti © ti. It is a straightforward calculation that for any evaluation 

t_5_ V I 

.12 J / 

thus if yl.t 1 © ti is equivalent to any A.tr it must be A.t^. 


context E and value u, ‘P'^J' (ti © tij v j = ^*P'^ = ‘P'^ (e 


Let E be the evaluation context E = let / = — [] in let x = / () in / (). 
We compute *P''^ (^E A.ti © f i j = ^ and ‘P'*^ (^E A.t^ ^ ^ showing that 

yl.ti © 11 is not equivalent to A.t_^. 

2.3 12 . . 

This example also shows that extensionality for expressions, as opposed to 
values, of function type does not hold. The reason is that probabilistic choice 
is a computational effect and so it matters how many times we evaluate the 
term and this is what the constructed evaluation context uses to distinguish the 
terms. 


A free theorem for lists Let r be a type and a not free in r. We write [r] for 
the type of lists y^a.(l+rxa), nil for the empty list and cons : Va.a —>■ [a] —> [a] 
for the other constructor cons = vl.Ax.Axs.f old (inr {x, x.s)). The function map 
of type Va.V/3.(a ^ /3) —)• [a] —>■ [/3] is the function applying the given function 
to all elements of the list in order. Additionally, we define composition of terms 
f o g as the term Xx.f{g{x)) (for x not free in / and g). 

We will now show that any term m of type Va.Vp.(a ^ /3) ^ [a] —>■ [/3] equiv¬ 
alent to a term of the form A.A.Xx.e satisfies m[][] (/ o g) ”^QD/ ° ®ap[][] g 
for all values f and all deterministic and terminating g. By this we mean that for 
each value v in the domain of g, there exists a value u in the codomain of g, such 
that gv =‘=* 2 ; Pqj. instance, if g reduces without using choice reductions and 
is terminating, then g is deterministic. There are other functions that are also 
deterministic and terminating, though, for instance Aa;.() © (). In the Appendix 
we show that these restrictions are not superfluous. 

So let TO be a closed term of type Va.V/3.(a —>■ /3) —)• [a] —>■ [/3] and suppose 
further that to is equivalent to a term of the form A.A.Xx.e. Let r, cr, p G T be 
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closed types and / S Val (cr —>■ p) and g € Tm (r —^ cr) be a deterministic and 
terminating function. Then 

0 I 0 h TO[]0(/og) =^‘"^m[][]/omap [][]5 : [r] [p]. 

We prove two approximations separately, starting with We use Theo¬ 

rem |4l8] multiple times. We have a, /3 | 0 h m[][] : (a ^ /3) ^ [a] —>■ [/3]. Let R = 
An.{('(;, u) \ gv u} be a member of VRel (r, a) and S G VRel (p, p) be the 
constant identity relation on Val (p). Let tp map a to i? and j3 to S. Proposi¬ 
tion |33] gives (w[][], m[][]) G |(a — >■ /3) — >■ [a] [/3]] ((p)^(n) for all n G N. 

We first claim that (/op, /) G [a ^ /3| (<p)(n) for all n G N. Since / is a value 
and has a type, it must be of the form Xx.e for some x and e. Take j G N, related 
values {v,u) G r(/), k < j and {E,E') G (k) two related evaluation contexts. 
We then have {E'[fu\) = {E'[f{gv)]) by Theorem 14.81 and the definition 

of relation R. Using the results about fpf (•) and (•) proved in Section E] in 
the Appendix this gives us 

< E E 2n(7r)«p^(U'[u;]) 


and the last term is equal to *P'^ {E'[f{gv)\) which is equal to fP'*^ {E'[f u]). 

From this we can conclude (m[][] (/ o p),m[][] /) G |[a] [/?]] (fp)^(n) for 

all n G N. Note that we have not yet used the fact that g is deterministic and 
terminating. We do so now. 

Let xs be a list of elements of type r. Then induction on the length of xs, 
using the assumption on p, we can derive that there exists a list ys of elements 
of type cr, such that map[][]pa;s ys and {xs,ys) G |[cr]] (<p)(n) for all n. 

This gives us (m[][] (/ op)a;s,m[][] f ys) G |[/3]]| ((p)^(n) for all n G N. Since 
the relation S is the identity relation we have for all evaluation contexts U of a 
suitable type, {E, E) G <5'^(n) for all n, which gives 

w[]D if°g)xs <"^mQ 0 /ps =^‘^m[][]/(mapO[]pxs) (m[][] / o map[][] p) xs 

where the last equality holds because /3-reduction is an equivalence. 

We now conclude by using the fact that m is (equivalent to) a term of the 
form yl.yl.Ax.e and use Lemma Hl^ to conclude m[] [] (/ o p) w[] [] / ° D 9- 
For the other direction, we proceed analogously. The relation for /3 remains 
the identity relation, and the relation for R for a is {(z;,u) I v gu}. 


6 Extension to references 

We now sketch the extension of F^’® to include dynamically allocated refer¬ 
ences. For simplicity we add ground store only, so we do not have to solve a 
domain equation giving us the space of semantic types and worlds [T] . We show 
an equivalence using state and probabilistic choice which shows that the addi¬ 
tion of references to the language is orthogonal to the addition of probabilistic 
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choice. We conjecture that the extension with higher-order dynamically allocated 
references can be done as in earlier work on step-indexed logical relations m- 
We extend the language by adding the type ref nat and extend the grammar 
of terms with i \ ref e | ei := 62 | !e with i being locations. 

To model allocation we need to index the interpretation of types by worlds. 
To keep things simple a world w S W is partial bijection / on locations together 
with, for each pair of locations (^ 1 ,^ 2 ) € /, a relation R on numerals. We write 
{£i,£2,R) S w when the partial bijection in w relates £i and £2 and R is the 
relation assigned to the pair {£i,£2)- Technically, worlds are relations of type 
Loc^ X V {{n I n G N}) satisfying the conditions described above. 

The operational semantics has to be extended to include heaps, which are 
modeled as finite maps from locations to numerals. A pair of heaps {hi, / 12 ) satis¬ 
fies the world ic, written (/ii,/i 2 ) G [u>J, when V(£i,t' 2 , & w, {hi{£i), h 2 {£ 2 )) & 

R. The interpretation of types is then extended to include worlds. The denotation 
of a type is now an element of W —>■ VRel (•, •) where the order on W is inclu¬ 
sion. Let WRel(T, t') = W VRel (r,T'). We define fA h ref nat] {ip){n) 
as \w. {{£i,£2) I (^ 1 ,^ 2 ,=) G w} where = is the equality relation on numerals. 

The rest of the interpretation stays the same, apart from some quantification 
over “future worlds” in the function case to maintain monotonicity. We also need 
to change the definition of the TT-closure to use the world satisfaction relation. 
For r G WRel (t, t') we define an indexed relation (indexed by worlds) as 

Vw' > WjVfc < n,V(hi,/i2) G ,Vui,U2 G r{w'){k), 
^l{{hi,E[vi])) <^H{h^,E[v 2 ])) 


r^{w){n) { {E,E') 


and analogously for A 

We now sketch a proof that two modules, each implementing a counter 
by using a single internal location, are contextually equivalent. The increment 
method is special. When called, it chooses, uniformly, whether to increment 
the counter or not. The two modules differ in the way they increment the 
counter. One module increments the counter by 1, the other by 2. Concretely, 
we show that the two counters pack (A — .ref 1_, Ax.la;, Aa;.() © {x := Six)) and 
pack (A — .ref 2, Xx.\x div 2, Ax.{) © (a; := S (S !a;))) are contextually equivalent 
at type 3a.(1 —>• a) x (a —5> nat) x (a —>■ 1). We have used div for the division 
function on numerals which can easily be implemented. 

The interpretation of existentials \A h 3a.r| {ip){n) now maps world w to 

, , I 3a, a' G T, 3r G WRel (a, a'), 

(pac u,pac v) \ g q, p .j-| Jq, 

To prove the counters are contextually equivalent we show them directly 
related in the value relation. We choose the types a and a' to be ref nat and 
the relation r to be Xw. {(£ 1 ,^ 2 ) | (^ 1 ,^ 2 , {(n. 2 ■ n ) | n G N}) G w}. We now 
need to check all three functions to be related at the value relation. 

First, the allocation functions. We only show one approximation, the other is 
completely analogous. Concretely, we show that for any n G N and any world w G 
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W we have (A — .ref 1, A — .ref 2) G |1 —>■ a] {r){w){n). Let n G N and w G W. 
Take w' > w and related arguments v,v' at type 1. We know construction 
that V = v' = {) so we have to show that (ref 1, ref 2) G |q;| (r) {w'){n). 

Let w” > w' and j < n and take two related evaluation contexts {E, E') at 
|a]| {r)^ {w"){j) and {h, h') G L^"J. Let (. ^ dom(/i) and E ^ dom(/i'). We have 

((/r, L;[ref 1 ])) = {{h \i ^ 1 ], L;[£])) 

and ((/i',L;'[ref 3 )) = ((/i' \l' ^ 2], E'[£'])). 

Let w'" be w" extended with {£,£', r). Then the extended heaps are in \ w"'\ 
and w'" > w". Thus E and E' are also related at w'" by monotonicity. Similarly 
we can prove that {£,£') G |a] (r)(j)(u;"'). This then allows us to conclude 
{{h [£ !->■ !],£'[£])) < {{h' [£' I-7- ‘^,E'[£'])) which concludes the proof. 

Lookup is simple so we omit it. Update is more interesting. Let n G N 
and w G W. Let £ and £' be related at Ja] (r)(w)(n). We need to show that 
({) (B {£ ■= S l £), 0 © {£' := S (S \£'))) G |1] (r)^{w)(n). Take w' > w, j < n and 
{h, h') G [w'J. Take related evaluation contexts E and E' at w' and j. We have 

{{h, E [() © (£ := S !£)])) = {{h, E [()])) + {{h, E[£:=S \£])) 

{{h', E' [() © (f := S S !f)])) = {{h', E' [()])) + iqj* {{h', E' [£' := S S !f ])) 

Since £ and £' are related at |q;] (r)(w)(n) and w' > w and {h,h') G [w'J we 
know that h(£) = m and h'{£') = 2 • m for some m G N. 

Thus {{h, E[£-=S\£]))= qi)- ((/ii, L;[()])) where hi = h[£ ^ m+ 1] . Also 

^^{{h',E'[£' ■=S S ]£']))= ^^{{h 2 ,E'[{)])) where /la = h' \£'^ 2-{m + 1) . 
The fact that hi and ft-a are still related concludes the proof. 

The above proof shows that reasoning about examples involving state and 
choice is possible and that the two features are largely orthogonal. 

7 Conclusion 

We have constructed a step-indexed logical relation for a higher-order language 
with probabilistic choice. In contrast to earlier work, our language also features 
impredicative polymorphism and recursive types. We also show how to extend 
our logical relation to a language with dynamically allocated local state. In 
future work, we will explore whether the step-indexed technique can be used for 
developing models of program logics for probabilistic computation that support 
reasoning about more properties than just contextual equivalence. We are also 
interested in including primitives for continuous probability distributions. 
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A Language definitions and properties 


T ::= a I 1 I nat | ri X r2 | n + r2 | ri —>■ r2 | /ia.r | Va.r | 3 a.r 

V ■.:= ® I {) I n I (wi, i’2) I Xx.e I ini v | inr v | Tl.e | packn 

e ::= * | () | n | (61,62) | Ax.6 | ini 6 | inr e | 71.6 | pack6 

I proj^ 6 I 61 62 I match (e, xi.ei, X2.62) | e[] 

I unpack ei as x in 62 | unfold e | fold6 | rand 6 
I ifi 6 then ei else 62 | P6 | S6 
i? :;= — I {E,e) \ (v,E) \ ini E \ inr E \ packi? 

I proj^ E\Ee\vE \ match (E, xi.ei, X2.62) | 

I unpack as x in e | unfold | foldiJ 
I ifi then 61 else 62 | randS | P | S i? 

Fig. 2 . Types, terms and evaluation contexts, n are numerals of type nat. 


a£ A 

- A\- 1 A\- nat 

Aha 


Z\ h ri Z\ h r2 
Zl h Ti X r2 


Zi h ri Zi h r 2 
Zl h ri + r 2 


Z\ h Ti Z\ h r2 zi, a h r Zi, a h r 

Zi h Ti ^ T2 Zi h 3 a.r Zi h Va.r 


Zi, a h r 
Zi h fia.T 


Fig. 3 . Well-formed types. The judgment Zi h r expresses ftvij) C Zi. 


The following lemma uses definitions from Section [3] 

Lemma A.l. <I> is monotone and preserves suprema of io-chains. 

Proof. Since the order in J- is pointwise and multiplication and addition are 
monotone it is easy to see that is monotone. 

To show that it is continuous let {fn}nGuj be an w-chain in jF. If e is a value 
the result is immediate. Otherwise we have 

<P (sup fn'] (e) = P' ( 

\n€uj / p \n^u) J 
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and since suprema in T are computed pointwise we have 

= P ■ sup {fn{e')) 


Using the fact that sum and product are continuous and that the sum in the 
definition of <P is finite we get 


<P 


sup fr, 

n^cu 


(e) = sup 

n^Lj 


p- fn{e') 


= sup<P{fn) (e) = sup^>(/„) (e) 


n^uj 


n^uj 


Example A.2. Let us compute probabilities of termination of some example pro¬ 
grams. 

— If V € Val then by definition tpl*' (?;) = 1. 

— If e € Tm \ Val is stuck then (e) = 0 by definition. 

— Suppose there exists a cycle e ei 62 e„ -i- e. Then tpl*' (e) = 

(ei) = • • • = qi* (e„) = 0. 

It follows from the assumption that none of are values and since the sum 
of outgoing weights is at most I we have that for each and e all other 
weights must be 0. We thus get that (e) = (ei) = • • • = tpl*- (e„) by 
simply unfolding the fixed point n-times. To show that they are all 0 we use 
Scott induction. Define 

5 = {/ e F\f{e) = /(ei) = fie^) = ... = /(e„) = 0}. 

Clearly S is an admissible subset of F and T € <S. Using the above existence 
of the cycle of reductions it is easy to show that S C ^ [<S]. Hence by the 
principle of Scott induction we have (•) G S and thus fP'^ (e) = IP''^ (ei) = 
... = ‘P^ (e„) = 0. 

This example also shows that we do really want the least fixed point of since 
this allows us to use Scott-induction and prove that diverging terms have zero 
probability of termination. 

Remark A.3. It is perhaps instructive to consider the relationship to the termi¬ 
nation predicate when we do not have weights on reductions. In such a case we 
can consider two extremes, may- and must-termination predicates. These can 
be considered to be maps Tm 2 where 2 is the boolean lattice 0 < 1. Let 
B — Tm —2. Since 2 is a complete lattice so is B. In particular it is a pointed 
w-cpo. We can define may-termination as the least fixed point oi E : B ^ B 
defined as 


f 1 if e € Val 

E{f){e) = < j., ,s , 

max/fe) otherwise 

k e-we' 
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Observe again that if e is stuck then W{f){e) = 0 since the maximum of an 
empty set is the least element by definition. 

Must-termination is slightly different. We need a special case for stuck terms. 

r 1 if e e Val 

!f-'(/)(e) = < min f{e) 3e' G Tm,p Gl,e ^ e' 

I e-we' 

[ 0 otherwise 

Let ], be the least fixed point of 'F and JJ. the least fixed point of F'. An 
additional property that holds for 4- and IJ-, because of the fact that 2 is discrete, 
is that for a given e, if e4.= 1 then there is a natural number n, such that 
<f'"(_L)(e) = 1, i.e. if it terminates we can observe this in finite time. This is 
because if an increasing sequence in 2 has supremum 1, then the sequence must 
be constant 1 from some point onward. 

In contrast, if (e) = 1 it is not necessarily the case that there is a natural 
number n with ^"(_L)(e) = 1 because it might be the case that 1 is only reached 
in the limit. 

The next lemma uses the abbreviation ; defined in Section [5l 

Lemma A.4. For all terms e, e' G Tm, (e; e') = *P'^ (e) • tP'*^ (e'). 

Proof. We prove two approximations separately, both of them by Scott induc¬ 
tion. 

< Consider the set 

/ < ‘P-^ (•) A Ve, e' G Tm, 1 

/(e;e')<‘P^(e)-*pne') j' 

It is easy to see that S contains T and is closed under w-chains, so we only 
need to show that it is preserved by F. The first condition is trivial to check 
since tP''^ (•) is a fixed point of F. Let f € T and e, e' G Tm. If e G Val then 
^(/)(e;e') = f{e') on account of one /3-reduction. By assumption f(e') < 
‘P'*^ (e') and by definition we have tP'*^ (e) = I. 

If e is not a value we have 4>(/)(e;e') = p ■ /(e";e') < EeAe" 3^ ’ 

(g//) . qj/l (g/) ^ qjJl (g) . (g/). 

Thus we can conclude by Scott induction that (■) G S. 

> For this direction we consider the set 

VE G Stk, e G Tm, v G Val, 1 
{E[e])>f{e)-Vp^ iE[v]) J' 

It is easy to see that it is admissible and closed under (p. Hence (■) G S. 

Thus we have, taking E = —;e' and any value v, that tP''^ (e) • (n; e') < 
iP"^ (e; e') and it is easy to see that iP'^ (n; e') = CP'^ (e'). 

Lemma A.5. Let e,e' G Tm. If e e' then for all k, (e) = fp^ (e'). 




s = UeE 
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Proof. When A: is 0 the result is immediate. So assume fc > 0. We need to 
distinguish two cases. 

— If there exists v' € Val such that e' v' then we also have e v' and 
we are done. 

— If not, then we need to inspect the definition of Red (e) and Red(e'). It 

is easy to see that any path tt € Red (e') corresponds to a unique path 
tt' • TT in Red (e). It is similarly easy to see that 2IJ (tt) = OH (tt' • tt) and that 
1{t^) = I (tt' • tt). Thus we have that (e) = (e'). 


Proposition A.6 . For each e G Tm we have (e) < sup^g^ (e)] . 


Proof. We use Scott induction. Let S be the set 


5= /g-F 


Ve,/(e) < sup (e)) 

k^uj ^ ' 


It is easy to see that S is closed under limits of w-chains and that _L G 5 so we 
only need to show that S is closed under F. Let f G S and e an expression. We 
have 


{ 1 if e G Val 

p ■ f (e') otherwise 

e-we' 

and we consider 4 cases. 

— e G Val. We always have e e and so we have that for any k > 0, 
(e) = 1 which is the top element. 

— e -S) e' and the reduction is not unfold-fold or choice. Then we use Lemma lT^ 
to get (e) = tp^ (e') for all k. Similarly we have that <?(/)(e) = /(e') from 
the definition of Thus we can use the assumption that f G S. 

— e e' and the reduction is unfold-fold. This follows directly from the defi¬ 
nition of Red (•), W and the assumption that f G S. 

— The reduction from e is a choice reduction. Suppose e reduces to ei, 62 , ..., e„. 
Then we know from the operational semantics that the weights are all 
We get 

n - 

^(/)(e) = X! ‘Pfe-n (e) = X! “‘Pfc • (2) 


Using the fact that (d) is an increasing chain in k for each a we have 

n 

sup hi (e)) = y] - sup hi {eh (3) 

kGuj '' ^ ^ n keLj ^ ^ 

By assumption f{ei) < sup/,g;^ ^tp^ (ci)^ for all f G {1, 2,..., n} which con¬ 
cludes the proof using ([2]) and ([3|). 
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Interpretation of types and the logical relation 

Lemma A.7. The interpretation of types in Fig.\^is well defined. In particular 
the interpretation of types is non-expansive. 

The substitution lemma is crucial for proving compatibility of existential and 
universal types. The proof is by induction. 

Lemma A.8 (Substitution). For any well-formed types A,a\- t and Z\ h cr 
and any (p we have |Z\ h T[cr/a]] {(p) = \A,a h r] {p [a i—?> |Z\ h (t] (</j)]). 

We state and prove additional context extension lemmas. The other cases 
are similar. 

Lemma A. 9. Letn G N. If{v,v') G |Z\ h ti —>• T 2 I {p){n) and {E,E') G |Z\ h T 2 I (p)^(n) 
then {E o {v []), E' o {v' [])) G |A h n] ((^)^(n). 

This follows directly from the definition of the interpretation of types. 

Corollary A.10. Letn G N. If{e,e') G |Z\ h ti| ((/j)^(n) and {E,E') G |Z\ h r2] {p)^(n) 
then 

[E o ([] e), E' o (Q e')) G [A h n ^ r2l (pfin). 

Proof. Let n G N. Take (v,v') G |A h ri —>■ T 2 ] (v 5 )(n). By Lemma [A.91 and 
monotonicity we have for all k < n, {E o (v W),E' o (v' [])) G |A h ri| {pY{k) 
and by the assumption that (e, e') G |A h ri| ((^)^(n) we have 

{E[ve])<^^{Eye']) 

concluding the proof. 

LemmaA.il. LetnGN. // (if, i?') G |A h r[/xa.r/a]| ((p)^(n) then 
{E o (unfold W),E' o (unfold [])) G |A h pa.rl (p)^ {n). 

Proof. Let n G N. We consider two cases. 

— n = m + 1 

Take (foldw, foldu') G |A h /xa.r| {p)(n). By definition 
{v,v') G |A h T[pa.T/a\l {p){m). 

Let fc < n. If fc = 0 the condition is trivially true (since (^^[unf oldfold u]) = 

0) so assume k = i + 1. Note that crucially i < m. Using Lemma 14.31 
Lemma 13.41 and Lemma 10 we have 

"Pfc (i;[unfold(foldu)]) =y {E[v]) 

< y iE'[v']) 

= (if^[unf old (f oldu^)]) 


concluding the proof. 
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— n = 0. This case is trivial, since ^Pq (e) = 0 for any e. 

Lemma A.12. Let n €N. If {E,E') S |A h /ra.T| ((^)^(n) then 

(E o (fold []), E' o (fold [])) e |A h r[/za.T/a]| (n). 

Proof. Easily follows from the fact that if {v, v') are related at the unfolded type 
then (foldu, foldu') are related at the folded type (using weakening to get to 
the same stage). 

To relate the logical relation to contextual and CIU approximations we first 
have that the composition of logical and CIU approximations is included in the 
logical approximation relation. 

Corollary A.13. If A \ E \- e e' : r and A \ E e' e" : r then 

A\rLe e" : r. 

This follows directly from the definition. This corollary in turn implies, together 
with Proposition 14.51 and the fact that all compatible relations are in particular 
reflexive, that CIU approximation relation is contained in the logical relation. 

Corollary A.14. If A \ E \- e e' : r then Z\ | U h e e' : r. 

Finally we have adequacy of the logical relation. 

Corollary A.15. Logical approximation relation is adequate. 

Proof. Assume 0 | 0 h e e' : r. We are to show that (e) < *P'^ (e'). 
Straight from the definition we have Vn G N, (e, e') G |0 h T]^(n). The empty 
evaluation context is always related to itself (at any type). This implies Vn G 
N, ip^J (e) < tP'^ (e') which further implies (since the right-hand side is indepen¬ 
dent of n) that sup„g^ (*P)^ (e)) < tP'*^ (e'). Using Proposition 13.51 we thus have 
fpl^ (e) < sup„g^ (tp)J (e)) < (e') concluding the proof. 

Lemma A.16 (Functional extensionality for values). Suppose T,a G T(A) 

and let Xx.e and \x'.e' be two values of type t ^ a in context A \ E. If for all 

u G Val (r) we have A \ E \- {Xx.e) u {Xx' .e') u : tr then 

A \ E \- Xx.e Xx'.e' : t ^ a . 

Proof. We use Theorem 14.81 several times and show Xx.e and Xx'.e! are logically 
related. Let n G N, v? G VRel(A) and ( 7 , 7 ') G |A h T] {(p){n). Let v = Xx.ej 
and v' = Xx'.e'Y. We are to show (v, u') G |A h r —>• cr| ((/?)^(n) and to do this 
we show directly (v, v') g\A\- t ^ cr| {ip){n). 

Let j < n, {u,u') G |r| {(p){n), k < j and {E,E') G |cr| {k). We have to 
show tp^ {E[vu]) < {E'[v'u']). From Proposition 14.51 we have that {v,v) G 

|t —>• cr] ((/j)^(n) and so {E[vu]) < *P'^ {E'[vu']). From the assumption of 
the lemma we have that vu' v' u' which concludes the proof. 
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Lemma A.17 (Extensionality for the universal type). Let r G T(Z\, a) be 
a type. Let A.e,A.e' be two terms of type \/a.T in context A \ F. If for all closed 
types a € T we have 

A \ r e e' : r[cr/a] 
then A \ r \- A.e A.e' : 'ia.r. 

Proof. We again use Theorem 14 . 81 multiple times. Let n € N, (p G VRel (A) and 
(7,7') G |Z\ h r] {(p){n). Let V = A.ej and v' = A.e'j'. We show directly that 
{v,v') G |Z\ I- Va.r] {p){n). 

So take a,a' G ^ and r G VRel ((T,cr') and we need to show (ey, e'7') G 
|A,a]| {p[a n- r])^(n). Let k < n and {E,E') related at k. We have to show 
[Elej]) < {E'[e'y]). From Proposition 14.51 we have 

(e 7 , ej') G |Z\, a]] {ip [a n- r])^(n) 

and so (i?[e 7 ]) < (i?'[e 7 ']). Let a be the types for the right hand side 

in p. Then E' G Stk (r[(f, cr'/Z\, a]). Using the assumption of the lemma we 
get that 67 ' < ^ e' 7 ' at the type t[u, <j' jA, a] which immediately implies that 

tP'*^ (i?'[e 7 ']) < (U'[e' 7 ']) concluding the proof. 

B The probability of termination 

We prove the claims from Section [S] about the termination probability. 

Proposition B.l. For any expression e, fP'^ (e) is a left-computable real num¬ 
ber. 

Proof. We first prove by induction that for any n, ^”(T) restricts to a map 
Tm —>■ [0,1] n Q. The proof is simple since the function T clearly maps into 
rationals and for the inductive step we use the fact that the sums in the definition 
of <1> are always finite, and the rational numbers are closed under finite sums. 

To conclude the proof we have by definition that CP'^^' (e) = sup^g^^ ^"'(T)(e) 
and we have just shown that all the numbers ^”(T)(e) are rational. Moreover 
the sequence {^"(T)(e)}^gpj is computable, since for a given n we only need to 
check all the reductions from e of length at most n to determine the value of 
<?"’(T)(e) and the reduction relation -w is naturally computable. 

Example B.2. To see that the probability of termination can also be non-computable 
we informally describe a program whose probability of termination would allow 
us to solve the halting problem were it computable. 

The program we construct is recursively defined as T = f ix[][], where 

p = Xf.Xx.tx © (17 © / (succ x)) 

where t a; is a program that runs the x-th Turing machine on the empty input 
and does not use any choice reductions. Thus fP'^ (tx) G {0,1}. It is well known 
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that the empty string acceptance problem is undecidable. Note that we put 17 in 
the program to ensure that every second digit in binary will be 0. It is an easy 
computation to show that 


= E 

n—0 

where = 1 if the n-th Turing machine terminates on the empty input and 0 
otherwise. If (T 1_) were computable we could decide whether a given Turing 
machine accepts the empty string by computing its index n and then computing 
the first 2 n digits of (T !_)■ 

We will now generalize the last example and show that any left-computable 
real arises as the probability of termination of a program. Technically, we show 
that given a term of the language that computes an increasing bounded sequence 
of rationals (represented as pairs of naturals) we can define a program that 
terminates with probability the supremum of the sequence. We then use the 
fact that our language F^’® is Turing complete to claim that any computable 
sequence of rationals can be represented as such a term of F^’®. 

Proposition B.3. For every left-computable real in [0,1] there is a program 
of type 1 —> 1 such that (e^ ()) = r. 

Proof. So let r : nat — >■ nat x nat compute an increasing sequence of rationals 
in the interval [0,1]. Additionally assume that for all n € N. 

rn=^ (^,C) 

for some kn,in € N. That is, r does not use choice reductions. This is not an 
essential limitation, but simplifies the argument which we are about to give. 

First we define a recursive function e of type e : (nat — > nat x nat) — > 1 as 
e = fix[][] (f where 


(fi = Xf.Xr.let {k,£) = rl in 
let y = rand£ in 
If y < k then () else f r' 


and 


, r (succ z) — (k,i) 

" ="" 1 - ■ 

and subtraction and division is implemented in the obvious way. Note that the 
condition in ip ensures that {k,£) does not represent the rational number 1 and 
therefore division would make sense. But technically, since we implement ratio¬ 
nals with pairs of naturals no exception can occur and we just represent the pair 
with the second component being 0 . 
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Let / and r be values of the appropriate type. We have 
fPi+i {^fr)< (0) + 


cf 

where rl {ki,£i). The inequality comes from the fact that applying r might 
take some unfold-fold reductions. Iterating this we get 


{er)<^ + - -P*(erCl) 

-vrj 'v-n ^ ' 


where rn (kn^i-n) 

(n) ^ , r(succ"z) - (fc„,4) 

■ l-an,4) 

is the n-th iteration of the ' used on r in Lp. 

It is easy to see that (e ) = 0 since it takes at least one unfold-fold and 
one choice reduction to terminate. Thus picking to = 1 we have ‘^ 2 + 2 n (e ’’) = 
and thus 


sup (e r) < sup 

nGoJ nGuj 

Using the same reasoning as above we also have 


h P — h 

cn-U- / \ \ 

(er) > — + —-- 

which shows (using Proposition [3^ that 




> 


k k 

sup (e r) < sup (e r) < sup 

n^u) Pn n^uj nGuj Pn 


and so 


sup ^ (er). 

n^uj Pn 


C Distributions 

We now define distributions and prove some of their properties and properties 
of the probability of termination which are used in the examples. 

By a distribution we mean a subprobability measure on the discrete space 
Val of values. Let 


Dist = {/:Val^[0,l]| ^ /(u) < 1} 

D^Val 
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be the space of subprobability measures on Val. To be precise, / S Dist are 
not measures, but given any / we can define a subprobability measure /r/(vl) = 
fi'^) given any subprobability measure /r, we can define G Dist 
as the Radon-Nikodym derivative with respect to the counting measure. Or in 
more prosaic terms /^(u) = ^ ({?;}). It is easy to see that these two operations 
are mutually inverse and since / € Dist are easier to work with we choose this 
presentation. 

Lemma C.l. Dist ordered pointwise is a pointed uj-cpo. 

Proof. The bottom element is the everywhere 0 function. Let {fn}nGuj be an 
w-chain. Define the limit function / as the pointwise supremum 

f(v) = sup/„(v). 

Clearly all pointwise suprema exist and / is the least upper bound, provided we 
can show that / G Dist. To show this last fact we need to show 

supfniv) < 1. 

but this is a simple consequence of Fatou’s lemma since from the assumption 
that {fn}neuj we have sup„g^ /„(-y) = lim„^oo /«(«) = liminf„^oo /«(«) and so 
by Fatou’s lemma (relative to the counting measure on Val) we have 


E 

D^Val 


sup/„(v) < liminf 

necj n-s-oo 



< liminf 1 = 1. 


Now define r: : (Tm — > Dist) (Tm Dist) as follows 


\6e if e G Val 

r;(i^)(e) = < ^ p. (p(e') otherwise 

y e-we' 

where Se is (the density function of) the Dirac measure at point e. Since Dist 
is an w-cpo so is Tm — s- Dist ordered pointwise. It is easy to see that in this 
ordering S is monotone and continuous and so by Kleene’s fixed point theorem 
it has a least fixed point reached in uj iterations. Let V = sup„gj^ (S'”(T)) be 
this fixed point. 

Lemma C.2. Let e G Tm and v G Val. IfV{e){v) > 0 then there exists a path 
TT from e to v, i.e. e steps to v. 

Proof. We use Scott induction. Define 

S = {f : Tm ^ Dist | Ve, v, f{e){v) > 0 ^ 37r, tt : e -w* w} 
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The set S contains _L. To see that it is closed under w-chains observe that if 
(sup„g;^ fn) (e)(u) > 0 then there must be n G w, such that /„(e)(u) > 0 so we 
may use the path from e to u that we know exists from the assumption that 
fn G S. 

It is similarly easy to see that given / G 5 we have S'(/) G S. Thus we have 
that V G S concluding the proof. 

Lemma C.3. For any expression e G Tm we have 

D^Val 

Proof. First we show by induction on n that all the finite approximations of 
(e) and V{e) agree. 

— The base case is trivial since by definition 

^ H«(T)(e)(u) = 0 = #(T)(e) 

•u^Val 

— For the inductive case we consider two cases. If e G Val then both sides are 
I. In the other case we have 

^ S"+i(T)(e)(u) = ^ I ^p-S"(e'))(u) 

t;eVal t;eVal ) 

= E (e^^--”( 0 (^)) 

«eval / 


by Tonelli’s theorem we can we can interchange the sums to get 


= E U E 

= ^p-<I>"(T)(e')=<?”+^(T)(e) 

P / 
e-we 


Thus we have that for all n, 

^ S"(T)(e)(u)=<I>"(T)(e) 

•u^Val 



sup (<I>"(T)(e))=<P*(e) 

n^LO 


and so 
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By the dominated convergence theorem we can exchange the sup (which is the 
limit) and the sum on the left to get 

sup[ ^ S"(±)(e)(^)| = ^ sup(S"(±)(e)(n)) 

Xti^Val / ti^Val 

= '^{e){v) 


as required. 


Proposition C.4 (Monadic bind for distributions). Let e G Tm and E 

an evaluation context of appropriate type. 

V{E[e])= Y. V{e){v)-V{E[v]). 

Proof. It is easy to show by induction on £ that 

Ve€Tm,E^{l.){E[e])= Y Y ^ M (4) 

f^Val 7T:e-^*v 
len(7r)<^ 

(using the fact that the length of the empty path is 0 and its weight 1). 
Similarly it is easy to show by induction on £ that 

Ve G Tm, (±) (e) (v) = Y ® (tt) (5) 

TTie-w*!) 

len(7r)<.^ 

which immediately implies 

Ve G Tm,T'(e)(v) = Y 2I^(7r) 

TTie-w*!) 

Using these we have 

V{E[e]) =supY E (tt) ■ {E[v]) 

i^OJ _-.r 1 * 

•u^Val TTie-w*!) 

len(7r)<.£ 


( 6 ) 
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and since for each v the sequence E 2IF(tt) • ien(7i-) ig increasing 

TTie-'^ V 
len(7r) <£ 

with i we have 

= E E 

1 £Guj * 

?;EVal Trre-w v 

len(7r)<^ 

= E E ^M-T^(.E[v]) 

■u^Val Trre-w*!! 

= ^ ViE[v]) ^ 2IJW 

i;^Val 7z:e^*v 

= ^ V{e){v)-V{E[v]) 

vG'Val 

Corollary C.5. Let e € Tm be typeable and E an evaluation context of appro¬ 
priate type. Then iE[e]) = >V(7r) ■ (£'H). 

Corollary C.6. For any term e and evaluation context E the equality 
VHE[e])= 'E{e){v)-^HE[v]) 

uGVal 

holds. 

Corollary C.7. Let e G Tm and E an evaluation context. Suppose 'D[e) = p-6y 
for some v € Val and p G [0,1]. Then {E[e\) = p ■ 

Proof. Use Proposition IC.4I and Lemma IC.3I 

Proposition C.8. For any evaluation context E and term e and any k gN, 

^UE[e])< Y. Win)-^^UE[v]) 

7r:e-w* v 

The proof proceeds by induction on k. 

D Further examples 

In this section we show further equivalences which did not fit into the paper 
proper due to space restrictions. 

Fair coin from an unfair one Given an unfair coin, that is, a coin that comes 
up heads with probability p and tails with probability 1 — p, where 0 < p < 1 we 
can derive an infinite sequence of fair coin tosses using the procedure proposed 
by von Neumann. The procedure follows from the observation that if we toss an 
unfair coin twice, the likelihood of getting (H, T) is the same as the likelihood 
of getting (T, H). So the procedure works as follows 
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— Toss the coin twice 

— If the result is (H, T) or (T, H) return the result of the first toss 

— Else repeat the process 


We only consider rational p in this section (for a computable p we could 
proceed similarly, but the details would be more involved, since the function 
which returns 1 with probability p and 0 with probability 1 — p is a bit more 
challenging to write). 

Let 1 < k < nhe two natural numbers and p = ^. Below we define Cp : 1 — >■ 2 
to be the term implementing the von Neumann procedure for generating fair 
coin tosses from an unfair coin tp which returns true with probability p and 
false with proability 1 — p. We will show that Cp is contextually equivalent to 
Ax.true © false. We define Cp as 

Cp = fix[][](p 

where 


2 

true 

false 


if e then ei else 62 
tv 


1 + 1 


inl() 
inr 0 

match (e, -e', ..match (e', ..false, ..true)) 
match (e, ..ei, .. 62 ) 

A().let y = randn in (p < k) 


and 


(fi = A/.A().let X = tp 0 in 
let y = tp{) in 
if X = p then / () else x. 

By a simple calculation using the operational semantics we can see that given 
any evaluation context E, we have iE[tp ()]) = (if[true]) + 2i^iP'f (E[f alse]). 
Given any value / of type (1 —>■ 2) and any evaluation context E with the hole 
of type 2 we compute that iP'*^ {E[ip f ()]) is equal to .|_ 2 ■ 

(i<;[true © false]). Finally for Cp and any evaluation context E with 
hole of type 2 we have 

‘P^ {E[ep ()]) = (pep 0) = + lE[ep ()]) 

+ 2 ■ (E;[true © false]) . 

from which we have by simple algebraic manipulation that iP'*^ {E[ep ()]) = 
tpl*' (i!i[true ® false]). 
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It is now straightforward to show 0 j 0 h Cp A().true © false : 1 2 

since both Cp and A().true © false are values, so we can show them related in 
the value relation. The proof uses reflexivity of 

Alternatively, we could have used Theorem l4.8l and showed directly that Cp () 
and true ©false are ClU-equivalent and then used extensionality for values to 
conclude the proof. 


A hesitant identity function We consider the identity function e that does 
not return immediately, but instead when applied to a value v flips a coin whether 
to return v or call itself recursively with the same argument. We show that this 
function is contextually equivalent to the identity function Ax.x. The reason for 
this is, intuitively, that even though e when applied may diverge, the probability 
of it doing so is 0. 

Example D.l. Let e = fix[][] (Xf.Xx.{x © f x)) : a —>■ a. We have 

a] 0 e Xx.x : a ^ a 


and 

a I 0 h Xx.x e : a —>■ a. 

Proof. We prove the two approximations separately. Let ip G VRel (a), n G N. 
Since e and Xx.x are values we show them directly related in the value relation. 
In both cases let (p = Xf.Xx.{x © f x) and h = Xz.6,p (foldi5,p)z. 

— By definition of the interpretation of function types we have to show, given 
k <n and {v,v') G pr{of){k), that {ev, (Xx.x) v') G pr{o()^ {k). 

cf 

It is straightforward to see that ev => pev using exactly one unfold-fold 
reduction. 

Now let {E, E') be related at k. We proceed by induction and show that for 
every i < k, {E[ev]) < {E'[v']) which suffices by Lemma [XT] When 

i = 0 there is nothing to prove. So let f + I. 

(A[eu]) = (pev) = {E[v © eu]). 

If = 0 we are trivially done. So suppose £' = £" +1 to get using Lemma [3^ 

{E[v ®ev]) = {E[v]) + (ev) 

Using the fact that < k and monotonicity we have 

iE[v])<^^ iE'iv']). 

Using the induction hypothesis we have 

‘PI// (ev) <<P^ (E'[v']) 
which together conclude the proof. 
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— Again by definition of the interpretation of function types we have to show, 
given k <n and {v,v') € ipr{a){k), that {{\x.x)v' ,ev) S ipr{a)^ {k). 

Again we have that ev' tpev' using exactly one unfold-fold reduction. 
Let ^ <k and {E, E') related at Using Lemma [SJ] and the fact that (•) 
is a fixed point of 'P we have 

iE'[ev'])=‘^^{E'[ipev']) 

= ^^^Ey]) + ^^HE'[ev']) 

and from this we get {E'[ev']) = {E'[v']) by simple algebraic ma¬ 

nipulation and thus {E'[ev']) = *P'^ {E'[v']). Using this property it is a 
triviality to finish the proof. 


D.l Further simple examples 

The following example is a proof of perfect security for the one-time pad encryp¬ 
tion scheme. Define the following functions 


not : 2 —2 

not = Ax.if X then false else true 
xor : 2 —>■ 2 —>■ 2 

xor = Ax.Ay.if x then not y else y 
gen : 2 

gen = true © false 

xor is supposed to be the encryption function, with the first argument the plain¬ 
text and the second one the encryption key. 

We now encode a game with two players. The first player chooses two plain¬ 
texts and gives them to the second player, who encrypts one of them (using 
xor) chosen at random with uniform probability and gives the result back to the 
first player. The first player should not be able to guess which of the plaintexts 
was encrypted. This is expressed as contextual equivalence of the following two 
programs 


exp = Ax.Ay.xor (x © y) gen 
rnd = Ax.Ay.gen 

To show exp rnd we first use extensionality for values so we only need 
to show that for all v,u € Val (2) 

xor (v © u) gen gen 
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and the easiest way to do this is by using CIU equivalence. Given an evaluation 
context E we have 


(£;[xor 


{v 0 u) gen]) = 


(spb (i5[xor V true]) + \ 
(i?[xor V false]) + 
(i?[xor u true]) + 
(i5[xor u false]) j 


and by the canonical forms lemma u and v can be either true or false. It is 
easy to see that the sum evaluates to 


-(2 • (£;[true]) + 2 • (£;[false])) 


quickly leading to the desired conclusion. 

If we had used the logical relation directly we would not need the canonical 
forms lemma, but then we would have to take care of step-indexing. 

A similar example is when in one instance we choose to encrypt the first 
plaintext and in the second instance the second one. Since the key is generated 
uniformly at random, the first player should not be able to distinguish those two 
instances. Concretely, this is expressed as contextual equivalence of the following 
two programs 


expj^ = Xx.Xy.xor x gen 
exp2 = Xx.Xy.xor y gen 

The proof is basically the same as the one above. Use extensionality and then 
CIU equivalence. 

D.2 Restrictions in the free theorem are necessary 

We show that the free theorem in Section [5] does not hold without the assump¬ 
tions on the behaviour of functions / and g. 

First, if / = (Ax.l.) © (Aa;.2), g is the identity function Xx.x and xs is the list 
[(), ()] then the term map[][](/ o g)xs can reduce to the list [1,2], however the 
term ((map[][] /) o (map[][] g)) xs cannot. The reason is that in the first case the 
reduction of / is performed for each element of the list separately, but in the 
latter case, / is first reduced to a value and then the same value is applied to 
all the elements of the list. Technically, the condition we need for / is that there 
exists a value /', such that / /', but this version is easily derived from the 

version stated above by congruence. 

Second, if g diverges with a non-zero probability for some value v, we take 
m to be the constant function returning the empty list and the list xs to be the 
singleton list containing only the value v. Then, if / is any value, to[][] (/ o g) xs 
reduces to the empty list with probability 1, however ((m-[][]/ o map[][] 5)) xs 
reduces to the empty list with a probability smaller than 1, since g is still applied, 
since we are in a call-by-value language. 
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Third, if 5 = Acc.l © 2, / is the identity function and xs is the singleton list 
containing () we take m to be the function that first appends the given list to 
itself and then applies map to it. We then have that m[][] (/ o g) xs can reduce 
to the list [1,2], but ((m[][]/) o (map[][](;)) xs cannot, since g is only mapped 
over the singleton list producing lists [1] and [2], which are then appended to 
themselves, giving lists [1,1] and [2,2]. 

And last, if m is not equivalent to a term of the form A.A.Xx.e then the term 
on the left reduces to two different (not equivalent) values (or even diverges), 
but the term on the right does not. We can use this to construct a distinguishing 
evaluation context. 

D.3 A property of map 

The result in Section [5] does not allow us to conclude 


mapQ [] if o g) map[] [] / o map]] [] g. 


for all / S Val (u — >■ p) and g G Val (r — >• cr), however we can show, using the 
definition of map, that this does in fact hold. By using extensionality fLemma l4.9l) 
we need to show for any list xs we have 


map]] [] if o g) xs (map]] [] / o map]] [] g) xs. 


If / and g are values, E an evaluation context and xs a list of length n, it is 
easy to see that 



(A'[map/xs]) T>(/Xi)(ui) 


where the first sum is over all the lists of length n and Xi and Ui are the Tth 
elements of lists xs and us, respectively. This then gives us that 


(i;[map/ (map^xs)]) 


is equal to 



On the other hand, we have that (A'[map (/ o g) xs]) is equal to 



and 
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° g) Xi){ui) = ^V(gxi){v) ■ V(fv)(u,) 

V 


together giving us 


E 



^V{gxi){v) ■V{fv){ui) 




which by Fubini’s theorem and the fact that lists of length n correspond to 
n-tuples, is equal to 


EE [YiC^ig j -^^{Eius]) 

US VS \i = l / 

which is the same as CP'^ (FI [map / (map g xs)]). 

If / and g are not equivalent to values, then the above result for map does not 
hold. Consider, for instance, / = \x.l © \x.2 and g the identity or conversely, 
when applied to the list xs = [(), ()]. The expression map[] [] (/ o g) xs can reduce 
to the list [1,2], whereas the expression (map[][] / o map[][] g) xs cannot. We can 
generalize this to show that if / is not equivalent to a value or g is not, then the 
stated equality does not hold. 
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x:t £ r A \- r 

A \ r h X : T 

A I r, x’.Ti \- e : T2 
A \ r Xx.e ■■ ti^T2 


Ah r 


A\rh {)-.1 


A \ r \- e\ ■. Ti A \ r \- 62 ■ r2 

Zi I rh (ei, 62 ) : Ti X T 2 


A \ r \- 6 : A \- T2 

A \ r \- ini e : ri + T 2 


A\r\-e: 


■ T2 


A \- T\ 


zi I n- inr e : ri + T 2 


A\ r,Xl-.T\\- e\ ■. T A\ r,X 2 ’-T 2 \- 62 ■ r A \ F h 6 T\ + T 2 

zl I _r h match (e, X\.6i, X 2 . 62 ) : r 

A,a \ r \- 6T A \ r \- 6Ti X t 2 a \ r 6 t' ^ t a \ r \- 6 : t' 

zi I _r h Zl.e : Vtt.r Zi | _r h pro j ^ e : r; Zi | _r h e : r 

Zi h ri A \ r \- 6 r[ri/a] 
zi I _r h packe : 3a.r 


A \ r \- 6 3a.n Zi h r A,Oi \ r,x ■. Ti\- 6' T 
A \ r \- unpack e as a; in : r 


A \ r \- 6 \ ^a.T 

Zi I h unfold e : r[/ia.r/a] 


Zi I f h e ; r[/ia.r/a] 
Zi I F h folde : /ia.r 


zi I f h e : Va.r Zi h 
Zi I r h e[] : r[r'/a] 


A \ r h 6 nat 
A \ r \- rand e : nat 


Zi I f h e : nat Zi|7~'l-ei:T Zi|7~'l-e2:T Zi|7^l-e: nat 

I J’ h if 1 6 then ei else 62 ■ T Zi | h P e : nat 

A \ r 6 nat 
Zi I h S e : nat 


Fig. 4. Typing of terms, where F 0 | F, x\t and Zi ::= 0 | Zi, a. 
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Basic reductions h—)- 

proj^ (ui, ti2)!-!->■ Ui unfold (fold v) 1-^ II 

{Xx.e)v !-!->■ e[u/2:] unpack (packu) as x in e e[u/x] 

(/l.e)[] I—!->■ e match(inl?;,xi.ei,X2.e2) ei[ii/xi] 

randni-^fc (fc G {1, 2 ,..., n}) match (inru, Xi.ei, X2-e2) e2[i’/x2] 

Pn !-!->■ max{n — 1 , 1 } S n i— w + 1 


ifi 1 then ei else 62 '—> ei 
One step reduction relation 

E[e] ^ E[e] 


ifi Sn then ei else 62 


62 


if 6 I 


Fig. 5 . Operational semantics. 
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x:t £ r 

A\r\-xnx:T A\r\- QTZQ :i 

A \ r \- ei TZ e'l : Ti A \ F \- €2 TZ '■ T2 A \ F, x-.t\ \- eJZ e : T2 

A \ F {61,62} 7 Z {ei, 62) : Ti X T2 A \ F \- Xx.e TZ Xx.e : ri — >■ r2 

A\F'r 6TZ6 :ti A \ F \- 6TZ e : T 2 

A \ F \- ini 6 TZ ini ; ri + T 2 Z\ | _r h inr e TZ inr e \ ti+ T2 

A \ F,xi:ti \- 61 TZ 61 ■. T A \ F, X2'-X2 62 TZ 62 ■ r A \ F \- 6 TZ e ■. ti-\-T2 

A \ F \- match (e, xi.ei,X 2 - 62 ) TZ match (e', a;i.ei, 0 : 2 . 62 ) : t 

A,a\ F \- 6 TZ 6 ■. T A\- Ti A \ F \- 6 TZ 6 ■. t[ti/ a] 

A \ F \- A .6 TZ A.e : Va.r A \ F \- (packe) TZ (packe^) ; 3a.r 

A \ F \- 61 TZ 61 ■. 3a.Ti Z\ h r Zi, a | r', a: : ri h e 7?. e' : r 
A \ F \- (unpack 61 as a; in e) 7?. (unpack 61 as a; in e^) : r 

Z\ I 7 ^ h e 7 ?, e' ; n X r2 Zi | 7 ^ h ei 7 J. e) : r' —> r Zi | 7 ^ h 62 7 J. 62 : r' 

Zi I 7 ^ h proj^ e TZ proj^ e : n Zi | 7 ^ h ei 62 72 . e) 62 : r 

Z\ I 7^ h e 72 e' : fia.r A \ F \- 6 TZ 6 : r[/aa.r/a] 

Z\ I 7^ h unfolde TZ unfolde^ : r[/ia.r/a] Z\ | 7^ h folde TZ folde^ : fia.r 

A\F\-eTZe: Ma.r , Zl I Th e 72 e' : nat 

- - - - - fivir ) C A - - - 

Z\ I 7^ h e[] 72 e [] : r[r /a] Z\ | 7^ h rand e 72 rand e : nat 

Z\ I 7^ h 6 72 6 ^ : nat Zl | 7^ h e 72 : nat 

Zi|7^|-Pe72Pe^: nat Z\|7^|-Se72Se^: nat 

Z\ I 7^ h 6 72 e' : nat Z\ | 7^, h ei 72 e) : r Z\ | 7^, h 62 72 62 : r 
Z\ I 7^ h ifi e then ei else 62 72 if 1 e' then e'l else 62 : t 

Fig. 6. Compatibility properties of type-indexed relations 
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{A h a| {(p) 
{A h nat| {ip){n) 

IA\-t X a] {ifi){n) 

|[Z\ h r + cr] ((p)(n) 

|[Z\ h r cr| {ifi){n) 
|Z\ h Va.rl {ifi){n) 

lA h 3a.Tj {‘fi){n) 

lA h fia.rj {ip){0) 
lA h p,a.Tl {ip){n+l) 


(Pr{a) 

{{k,k) I fe e N,fc > 0} 


{{v,u), {v',u)) 


U { (inr V, inr v' 
(Ax.e, Xy.e 


{v,v') e [Zi h r] {ip){n), 

G [Zi h al (</9)(n) 

{(ini n, ini v') \ {v,v') € lA h rj {(p){n)} 

(v,v') € |[Z\ h cr| (</9)(n)} 

Vj < n,V(v,v') G |[Z\ h r| (ip)(j), 
((Ax.e) V, (Xy.e') v') G [Zi h all (ip) 


(J) 


(A.e, A.e') 


Va,a' G T,Vr G VRel (a, a') , 

(e, e') G |[Z\, a h t} (<p [a h-)- r])^(n) 


1^ (pack V, pack v') 


3a, a' G 1, 3r G VRel (a, a'), 

(v, v') G |Z\, a\- T\(ip[a r]) (n) 


Val ((pi(/^a.r)) x Val (</92(AtQ!-'r)) 

{ (foldn, foldn') | (v, v') G |Z\, a\- T\(ip[a \A\- y,a.T\ (ip)]) (n)} 


Fig. 7. Interpretation of types. 


if ei 


dx yT., dx y-p.. y-J... C^/X 

e © e = e ei © 62 = 62 © ei e © 1/ < e 

,1 dx •r y-rs dx ,1 dx 

62 then ei = 62 it ei © 62 = ei then ei = 62 


Fig. 8 . Basic properties of and We write 17 for any diverging term (i.e. 

(17) = 0) and e©e' as syntactic sugar for ifi rand2 then e else e'. Note that the 
choice when evaluating e © e' is made before e and e' are evaluated. 



